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Abstract} - 

Introduction. A model to explore the relations among local and global relevance-based information behaviour is proposed that is based on objective and subjective measures of the relevance of the Website 
contents. 

Method. Global interest for the Website was researched using data on visits, while local use was explored with two surveys on the motivations and self-assessed knowledge of a group of fifteen students, 
administered to them before and after being exposed to the information site. 

Results. Results suggest that improvement in competence perception and the number of visits are, in general, measures of the site relevance that offer similar results. Regarding general data, the most 
visited competences are those of a more general interest (such as learning to learn), and the students show a big pre-test confidence in their knowledge of most of the studied competences. 

Conclusions. The increase in the subjective level of competence in an e-learning site is a good predictor of the whole success of a site, when working with their primary intended audience. In the future, to 
have independent measures, it would be interesting to carry out expert evaluation of the real competences of the students. 


CHANGE FONT 


Aims and context 

This study intends to explore the similarities and differences in the behaviour of the large audience of an e-learning Website, Alfin-EEES, on the one hand, and, on the other, a local subset of its main theoretical constituency of users, in 
this case, first-year university students. 

In a more abstract sense, the study tries to offer insights into the relation between the macro-environment of a Website, its global audience, and the specific use that a well-defined group of target users would make and is making of 
the system: a micro-environment. Last but not least, it provides a model to study the relation between relevance and other variables like use, expectations and motivation in an e-learning site. Such studies can be interesting both for 
theoretical and practical reasons, as presented below. 

Theoretical context 

Relevance is one of the key aspects in information retrieval, and has been the object of extensive research. It could not be otherwise, because relevance is the ultimate measure of the effectiveness of an information system. Relevance 
has been the topic of an enormous corpus of research with very different approaches (see Saracfli 2007a, I|07b, for a recent review), some dealing with users' judgments and other more related to information systems architecture 
or document contents. Also, many constructs have been identified behind relevance that are useful to explain it, such as topicality, novelty and reliability. Eleven different factors were identified in a single study (Wang and Soergel 
1998; Wang and White 1999) . 

Of the many approaches possible, this paper is interested in the relation between gross sociological measurements of the relevance of a Website (like the number of visits recorded in Web logs) and psychological measures of relevance 
provided by selected target audiences and that are related to their motivations and expectations. 

In particular, this paper tries to research the relationships among three very different groups of measures related to the relevance of Website contents, with the peculiarity that the first is objective and the other two are subjective, 
and intend to capture a cognitive change in the user after being exposed to a Website. 

• The first is the number of visits that each site, section or page receives. It is supposed that, under normal conditions, more visits imply that some content is considered more relevant, an approach that, of course, is widely 
used. 

• The second is a user's judgement of the relevance of a section of the Website to their professional future and that of its peers. 

• The third is a self-estimation of the user's knowledge of this topic. 

The last two measures can be combined in different aggregations. When compared with the number of visits, only one of them offered consistent results: that consisting in the difference between the perception of competence before 
and after a preliminary but comprehensive use of the site, or, in other words, the change in the self-assessed competence after being exposed to a Website. This was an empirical relation found after reviewing different results 
regarding the relevance assigned by a group of students to the contents of the Website and a self-assessment of their relative competence in them, before and after being exposed to the Website. 

The resulting theoretical model is expressed in Figure 1. The two user aspects that are explored regarding relevance are a) expectations as a result of previous knowledge and b) motivation that results form the perception of a 
knowledge gap. 

Relevance is seen as eliciting the three different kinds of responses that are studied: relevance judgements, self-competence judgements and the use of the Website. It is supposed that if an item is judged more relevant after using the 
information, this is related with a growing sense of the previous knowledge gap, because previous knowledge remains constant and, therefore, the relevance assigned to this piece of information increases. 
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Figure 1: A model of the studied variables 

(Some lines are dashed to avoid confusion where they cross others.) 


Applied research 

This research was very interested in contributing to a model of how relevance judgments are affected by user behaviour, but also practical implications arise. 

The relation between the global and local use of information is a very important question to develop fine models for Web site evaluation. This is a general problem in Web-based information systems and, consequently, also a key 
question for e-learning sites and, more specifically, for AlfinEEES, a portal that intends to offer relevant information in the specific field of generic and information competences. 

It is a general problem because a Website is a project subjected to the laws of economy. It has a demand and it is by fulfilling this demand in an effective and efficient way, that it will get the resources it needs to thrive. 

But on the other side, a Website, by its very nature, has a world-wide audience and this opens a door to a potential paradox: it might be that a Website is being successfully visited by a theoretically sufficient number of users, and that, 
on the other hand, it is failing to address its real intended audience, the one that makes it a feasible project. 


Organization of the paper 

This paper explains the results obtained in the research experiment and discusses the main conclusions that were reached. It is organized in four parts: first, the case study Website is briefly described and explained, attending to its 
aims, philosophy and content architecture; secondly, the methodology of the study is presented, which is summarily based on the comparison between Weblogs, on one side, and some relevance-related tests with a small survey of first 
year students; thirdly, the results about this relationship are offered; and, finally, the results are discussed, the main conclusions are summarized; and some perspectives for future research are offered. 


The target Website AlfinEEES: a brief explanation 

Alfin-EEES {?\w\a 2005,2006) stands for A/fabetizacidn Informational en el Espacio Europeo de Education Superior, that is, Information Literacy in the Higher Education European Space. It is a Website developed by a team directed by 
Maria Pinto with a research grant from the Spanish Ministry of Education. 

The portal was designed to provide online training in information competences for higher education students with a wider scope than usual, inserting information competencies into the frame of the development of generic 
competences, a key aspect of the Tuning project and the Bologna process (Gonzalez and Wagenaar 2003) . 

Alfin-EEES is organized hierarchically in competencies and sub-competencies. From the home page the user can also access the presentation of the project, the mention of credits and authorship and the support tools. The presentation 
explains the underlying philosophy, objectives, methodology and contents. The support tools are of different types: a) information seeking tools—the map of the site and the search engine; b) help in using the site—help and contact; 
c) study support materials—glossary, list of general resources (the specific resources appear in each of the sub-competencies) and in the future, FAQs; and d) for communication—forums, e-mail and chat rooms. 

The main part of the content architecture is the taxonomy of the six competencies, divided into their sub-competencies, which are: 

• Learning to learn: the concept of learning to learn is explained, how to learn to learn, how to be autonomous and in charge of our own learning process. 

• Learning to seek and assess information. 

• Learning to analyse, synthesize, and communicate: how to read better, how to segment the information to subsequently reorganize it using the techniques of outlining, graphic representation and summary and how to 
communicate the new knowledge in writing and using graphic presentations while respecting the contribution ofauthors whose ideas have been used. 

• Learning to generate knowledge: in-depth study of the processes of creation and innovation, of the principles of scientific thought and of the techniques for organizing projects, to train the student in the principle phases of 
knowledge creation: creation, research and development. 

• Learning to work with others: the ethical bases for co-existence and teamwork are posed, as well as how to recognize and approach conflict, using win-win negotiation techniques. 

• Using technology to learn: operational systems, office computer applications, communication tools and the e-learning environments most often used in universities are introduced. 

The competencies and sub-competencies are related to the knowledge production cycle, documentation, analysis, synthesis, communication and to its sub-contexts: working within a community and often as part of a group, 
contemplated in turn within the objective of educational intervention, which is teaching to learn to learn at the university level (Figure 2). The screen with the whole set of categories is presented in Figure 3 and its translation is 
available in Table 2. 
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Figure 3: Competences in Alfin-EEES 

The information offered for each competence and sub-competence is standardized. A basic chart outline for each compentence-subcompetence is shown in Table 1, and an example is presented in Figure 4, specifically a screen 
corresponding to the entry page of the sub-competence 'Reading to learn'. 
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Graphic description (maps, schemes, etc.) 
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Advice (observations for lecturers and students) 


Table 1: Outline of the structure of the information 
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• OPTIMIZAR nuestro aprendizaje. 

La lectura es, por tanto, un instrumento fundamental dentro de ese proceso de aprendizaje y requiere una actitud reflexiva, 
crftica y activa. 


OBJETIVOS 


Un estudiante universitario, a la hora de enfrentarse con un texto, debe ser capaz de conseguir los siguientes objetivos: 

• SER CAPAZ de expresar el contenido fundamental de la informacion lefda. 
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• SER CAPAZ de integrar y relacionar la nueva informacibn con el conocimiento ya adquirido. 
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Figure 4: The 'Reading to learn' competence homepage 


Methods 


Fortunately the highly structured nature of the portal content allows for a precise recounting of the use of the Website and, subsequently, an easy correspondence between this information and the data regarding the users' attitudes 
and self-assessment results on their information literacy sub-competences. This one of the reasons for AlfinEEES serving as a test bed for this study. 

First, the global interest for the Website was researched using the site traffic logs, with Google Analytics. As Alfin-EEES'^ highly structured, each page or group of pages corresponds to each competence or sub-competence, as explained 
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above. So, it is very easy to compare the interest of the global audience in each competence and use this data to make comparisons. 

Secondly, two surveys were carried out on a group of nineteen students of the first year of the undergraduate library and information science programme of the University of Saragossa during November 2009, with the aim of 
evaluating their interest in each competence and their statements about their subjective level of competence, both individually and in relation to their group. The first survey was carried out on 11 th November before being exposed to 
the J/f/n-fffSVI ebsile, as a pre-test condition. The second survey, using exactly the same items, was carried out on 29 th November after they have extensively used the site, as a post-test condition. 

The students were asked about the importance of each competence and sub-competence for their professional future and their personal opinion about their level of competence in each topic and about the level of competence of their 
whole group. They were asked also about their overall competence in information technologies and information management. All the questions were measured with a five-point scale. A reproduction of the survey Webpage is offered in 
Figure 5 and is available at the Alfin-EEES Website . The topics hove been translated to English in Table 2. 

To simplify, trying not to confuse them, the students were asked only for the sub-competences, assuming that the competences could be calculated as a mean of their sub-competences. This was proved partially untrue when a control 
question was introduced on 'Learning to analyse information'. This is a very interesting finding and a prospective hypothesis to explain it can be advanced: it is probably so because the competences are complex constructs that do not 
match exactly the sub-competences for all the population, so that, in practice, the designers of the systems and the users might have different concepts about them. 
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Figure 5: The survey Webpage 


The research model and method 

Figure 6 summarizes the logic of the research model. The intended aim of the /t/fi/j-fffSWebsile is to improve the information competences of university students. It is used by a wide global audience, as showed by Google Analytics, 
that cannot be stratified accordingly to the initial target audience. It is supposed that the more a Web page is used, the more relevant is for a community or various communities of users; and that this measure has significance inside 
the site, that is, the more a page is visited in relation to the rest, the more relevant it is, and vice versa. 

To study the intended audience, a survey was applied to a group of university students. The survey intends to establish the interest of the students in each competence (and sub-competence) by asking them about the importance of 
each item for their future. So, an indicator of relevance was obtained for each competence and sub-competence. Besides that, it asks the students about their level of knowledge on each competence, trying to establish if it is related to 


http://informationr.net/ir/15-4/paper453.htinl 
























Local versus global information relevance in Website use: a case study with the information literacy portal AlfinEEES 


Page 7 of 11 


their interests. This survey is done before they know Alfin-EEES. This information serves as an estimator of the existing gap in their knowledge. 

Later, they were exposed to Atfin-EEESmA the test was applied again to get information about how their new knowledge on information and generic competences had affected their perception of the importance of each item. 

As may be seen in Figure 6, the studied variables are considered expressive of the basic constructs that are studied: user motivation, which increases or decreases the assignment of relevance to an information item (e.g., a Web page 
on an information literacy competence), and eliciting a using behaviour. As a result of this behaviour, users re-evaluate both their interest in the information and their competence and the relevance of this item changes accordingly. 

In the future, a measurement of the actual competences would be needed to complement data based on the students' self-reports. In this way, we will be able to obtain information in both directions: a) how the objective IL tools may 
diverge from students' opinions on their competence; and b) how students' self-perception differs from the standard consensus among information specialists. The research team has developed a tool for the online assessment of 
information competences and is conducting the first tests. 

Finally, it would be interesting to research a black point in the model, that is, how student interests affect their exposition to information literacy information. This could be done, for example, measuring the time that students spend in 
each page of the Website or how many times they access the different Web pages. 



Figure 6: Design of the research model 

Macro-analysis: using visit statistics to understand the users' interest 

To get a glimpse of the topics of interest to the >4///0-/if£faudience, the Google Analytics data were normalized, obtaining the percentage of visits to each competence and their sub-competences since 2005, and also an index that 
compares the visits per page in each item with the mean of visits per page for the whole site (Table 2). 


Categories 

Visits 

Pages 

Visits per 
page 

Index 

% 

Learning to learn 

3,399.00 

1 

3,399.00 

3.56 

2.28 

Ability to learn 

9,577.00 

7 

1,368.14 

1.43 

0.92 

Autonomous learning 

14,681.00 

7 

2,097.29 

2.20 

1.41 

Subtotal 

27,657.00 

15 

1,843.80 

1.93 

1.24 


Learning to search and assess 
information 

1,542.00 

1 

1,542.00 

1.62 

1.04 

Learning to search 
information 

6,874.00 

7 

982.00 

1.03 

0.66 

Learning to assess 
information 

2,773.00 

7 

396.14 

0.42 

0.27 

Subtotal 

11,189.00 

15 

745.93 

0.78 

0.50 


Learning to analyse, 
synthetize and communicate 

4,187.00 

1 

4,187.00 

4.39 

2.81 

Learning to analyse 

14,142.00 

1 

14,142.00 

14.82 

9.50 

Reading to learn 

2,703.00 

7 

386.14 

0.40 

0.26 

Learning to segment 

14,491.00 

7 

2,070.14 

2.17 

1.39 

Subtotal 

35,523.00 

15 

2,368.20 

2.48 

1.59 


Learning to synthetize 

2,104.00 

1 

2,104.00 

| 2.21 

1.41 
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Learning to schematize 

8,299.00 

7 

1,185.57 

1.24 

0.80 

Learning to abstract 

5,620.00 

7 

802.86 

0.84 

0.54 

Subtotal 

16,023.00 

15 

1,068.20 

1.12 

0.72 


Learning to communicate 

1,143.00 

1 

1,143.00 

1.20 

0.77 

Learning to write 

7,776.00 

7 

1,110.86 

1.16 

0.75 

Learning to cite 

3,562.00 

7 

508.86 

0.53 

0.34 

Learning to present 

7,981.00 

7 

1,140.14 

1.20 

0.77 

Subtotal 

20,462.00 

22 

930.09 

0.97 

0.62 

Subtotal "analyse, etc," 

76,195.00 

53 

1,437.64 

1.51 

0.97 


Learning to create knowledge 

1,368.00 

1 

1,368.00 

1.43 

0.92 

Innovation and creativity 

5,429.00 

7 

775.57 

0.81 

0.52 

Research abilities 

6,413.00 

7 

916.14 

0.96 

0.62 

Project management 

4,326.00 

7 

618.00 

0.65 

0.42 

Subtotal 

17,536.00 

22 

797.09 

0.84 

0.54 


Learning to work together 

741.00 

1 

741.00 

0.78 

0.50 

Ethics 

2,307.00 

7 

329.57 

0.35 

0.22 

Learning to work in group 

3,298.00 

7 

471.14 

0.49 

0.32 

Decision-taking and 
negotiating 

4,315.00 

7 

616.43 

0.65 

0.41 

Subtotal 

10,661.00 

22 

484.59 

0.51 

0.33 


Using technology to learn 

1,129.00 

1 

1,129.00 

1.18 

0.76 

Operative systems 

4,521.00 

7 

645.86 

0.68 

0.43 

Ofimatic suites 

6,024.00 

7 

860.57 

0.90 

0.58 

Communication tools 

2,762.00 

7 

394.57 

0.41 

0.27 

Learning environments 

2,335.00 

7 

333.57 

0.35 

0.22 

Subtotal 

16,771.00 

29 

578.31 

0.61 

0.39 


Total 1148,820.001 156 | 953.97 | 1.00 0.64 


Table 2: Basic use statistics of Alfin-EEES 

Figure 7 shows the mean of the visits a page for each whole competence. The category that rises more interest is 'Learning to learn', which is understandable, as it is the most interdisciplinary, so with a bigger potential target 
audience, and a key topic of interest in pedagogy. The same can be said of the second and the third, 'Learning to analyse, synthesize and communicate information', and 'Learning to create knowledge'. Very near is the fourth category, 
which is the most specific of the library and information science field: 'Learning to search for and assess information'. The one that is less retrieved is 'Learning to work together'. 


0,00 

500,00 1.000,00 

1.500,00 2.000,00 

Learning to learn 


_J 1.843,80 

Learning to search and assess information 

_J 745,9 

'3 



Learning to analvze. svnthetize and communicate 



| 1.386,64 


Learning to create knowledge — 

- 1 797, 

,09 



Learning to work together — 

484,59 




Using technology to learn _ 

578,31 





Figure 7: Visits per page to each competence 

The sub-competence level offers a more granular perspective ( Table 2) . The topics that attract more interest are 'Autonomous learning', 'Learning to segment', 'Learning to analyse' and 'Learning to learn'. On a page-by-page basis, 
'Learning to analyse' is, by far, the most visited; and the least visited is 'Ethics'. 

Micro-analysis: assessing the motivations of a group of freshmen 

The results of the two surveys with the freshmen from the University of Saragossa are provided in Table 3. The importance and subjective assessments of their competence and that of their group's are offered before (pre-test) and 
after (post-test) working with Alfin-EEES. The differences between their competence and the importance they give to each sub-competence have been calculated and also the differences before and after the test. 
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Importance 
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pages 












Ability to learn 

0.94 

4.74 

3.74 

3.84 

-1.00 

-0.11 

4.74 

3.63 

3.79 

-1.11 

-0.16 

0.00 

Autonomous 

learning 

1.44 

4.37 

3.47 

3.74 

-0.89 

-0.26 

4.42 

3.53 

3.58 

-0.89 

-0.05 

0.05 

Learning to 

search 

information 

0.67 

4.79 

3.21 

3.68 

-1.58 

-0.47 

4.42 

3.89 

3.79 

-0.53 

0.11 

-0.37 

Learning to 
assess 

information 

0.27 

4.74 

3.26 

3.47 

-1.47 

-0.21 

4.37 

3.32 

3.68 

-1.05 

-0.37 

-0.37 

Reading to 
learn 

0.00 

4.53 

3.89 

3.79 

-0.63 

0.11 

4.58 

4.16 

3.89 

-0.42 

0.26 

0.05 

Learning to 
segment 

1.42 

3.84 

3.00 

3.53 

-0.84 

-0.53 

4.11 

3.53 

3.58 

-0.58 

-0.05 

0.26 

Learning to 
schematize 

0.81 

4.63 

3.42 

3.68 

-1.21 

-0.26 

4.47 

3.68 

3.68 

-0.79 

0.00 

-0.16 

Learning to 
abstract 

0.55 

4.68 

3.74 

3.79 

-0.95 

-0.05 

4.58 

3.79 

3.68 

-0.79 

0.11 

-0.11 

Learning to 
write 

0.76 

4.53 

3.68 

3.53 

-0.84 

0.16 

4.89 

4.05 

3.95 

-0.84 

0.11 

0.37 

Learning to 
cite 

0.35 

4.26 

3.37 

3.68 

-0.89 

-0.32 

4.26 

3.37 

3.53 

-0.89 

-0.16 

0.00 

Learning to 
present 

0.78 

4.53 

3.42 

3.58 

-1.11 

-0.16 

4.74 

3.79 

3.63 

-0.95 

0.16 

0.21 

Research 

abilities 

0.63 

4.58 

3.00 

3.37 

-1.58 

-0.37 

4.26 

3.16 

3.37 

-1.11 

-0.21 

-0.32 

Project 

management 

0.42 

4.53 

3.11 

3.42 

-1.42 

-0.32 

4.16 

2.95 

3.00 

-1.21 

-0.05 

-0.37 

Ethics 

0.23 

4.42 

4.05 

3.89 

-0.37 

0.16 

4.37 

3.79 

3.79 

-0.58 

0.00 

-0.05 

Learning to 
work in group 

0.32 

4.58 

3.79 

4.05 

-0.79 

-0.26 

4.63 

3.74 

3.95 

-0.89 

-0.21 

0.05 

Decision-taking 

and 

negotiating 

0.42 

4.84 

3.79 

3.68 

-1.05 

0.11 

4.58 

3.63 

3.74 

-0.95 

-0.11 

-0.26 

Operative 

systems 

0.44 

4.53 

3.79 

3.68 

-0.74 

0.11 

4.32 

3.84 

3.53 

-0.47 

0.32 

-0.21 

Ofimatic suites 

0.59 

4.53 

3.53 

3.42 

-1.00 

0.11 

4.53 

3.42 

3.53 

-1.11 

-0.11 

0.00 

Communication 

tools 

0.27 

4.58 

3.68 

3.42 

-0.89 

0.26 

4.63 

3.68 

3.58 

-0.95 

0.11 

0.05 

Learning 

environments 

0.23 

4.42 

3.21 

3.37 

-1.21 

-0.16 

4.11 

3.47 

3.79 

-0.63 

-0.32 

-0.32 


Table 3: Surveys with the first-year students of the University of Saragossa 

The students seem to think that all the sub-competences proposed are very important for their future, as they give them scores of more than '4'. It is curious, but not surprising, in a world that is becoming more social and 
interconnected, that they give the maximum punctuation to learning 'Decision-taking and negotiating'. The least punctuation is for 'Learning to segment', which they probably see as very peripheral or do not even understand. 

In general, the students have a very positive impression of their competences in information management and other related generic competences, and data does not show great differences between their own assessment and the one 
they do of their peers'. They show a great confidence in the materials that they are offered, and they think that they have improved quickly in their competence after reading and using them, as it can be inferred from the punctuation. 

Results could be very conditioned by local experiences. For example, the significant difference in 'Learning to search for information' could be affected by strong motivation and fresh new knowledge on the topic, because this is a 
subject they were just trying to pass at the moment of the survey. The same can be said of their struggling with digital 'Learning environments'. 

After their exposition to Alfin-EEES, 'Learning to write' and 'Learning to work in group' become more important. This is probably also a local effect, as these aspects are very important in the teaching methodology that is followed in the 

library and information science undergraduate programme of the University of Saragossa. 

Comparing both groups of data 

After extensively reviewing line graphics for the different sets of data, it can be affirmed that there is a general lack of correlation in the information offered by page visits and the relevance and competence data offered by the group 
students. Only in the general competence related to autonomous learning and the general ability to learn, a great correlation can be observed. (A recent study, with a completely different methodology, found that simple 'user 
relevance rankings had little or no predictive power about the actual use of a Web page to solve a problem (Coiera and Vickland 2008) 

From this lack of connection, it seems clear that the Website is used by other audiences that give, for example, a great importance to learning to learn, autonomous learning, and segmenting and schematizing information. 

There is, in any event, an important exception to this lack of correlations. It affects the comparison between the percentage of visited pages per category and the improvement in the perception of competence before and after the use 
of Alfin-EEES, as it is showed by the difference in 'My competence' after the exposition to the e-learning Website (Figure 8). If the number is positive, that means that the subjective sense of competence of the students has grown after 
using the system, and vice versa. 

This difference of subjective competence before and after using the system can be considered a measure of the value of the materials offered in the Website (relevance), and it certainly correlates (with some exceptions) with the visits 
that the sub-competence Web pages are receiving. It is remarkable that this correlation is not working at all in the topics related to technology, which are not the stronger part of the contents (mainly because their rapid obsolescence), 
a fact that might be in fact strengthening the hypothesis. 
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% visits ■ difmycomp. 



Figure 8: Percentage of visited pages per category versus the improvement in the perception of competence before and after the use of Alfin-EEES 


Conclusions and future work 


The strong relation between the shape of the percentage of visits, which was postulated as a measure of relevance at the macro-level, and the change in the subjective perception of personal competence after being exposed to the 
Website, a measure of the information gap that has been covered by the site, supports the hypothesis that the increase in the subjective level of competence in an e-learning site is a good predictor of the whole success of a site, when 
working with their primary intended audience. In this dimension, a strong and demonstrable relation between the macro/global and micro/local levels of analysis appears to exist. 


From a theoretical point of view, this is just another confirmation of the relation between the information gap on the part of the user, measured throughout the process of knowledge improvement, and the information relevance on the 
part of the collection. From this perspective, relevance appears to be something connected with the perceived information gap, even a function of it. 

Besides that, the study provides further insights in the relation between relevance and use of a Website, on the one hand, and the expectations and motivations of the users, on the other hand; and a model to study them. 


Taking into account a more practical and applied perspective, the change in the subjective perception of personal competence indicator is interesting because it is not a direct or intrusive one, and, in consequence, it is less prone to be 
manipulated by the users. As a result, it could be used as a relatively inexpensive non-intrusive pre-test of the relevance of a site or as a complementary evaluation tool. 


An important result of this study is that the correlation was clearly affected by the quality of the materials (technological pages get quickly out of date) and the existence of different potential audiences for a Website, in this case not 
only library and information science undergraduatess but probably also pedagogy and school and high school teachers and students. For these reasons, the model should be enriched in the future to deal with the quality of the 
collections and target group segmentation. 

It is a clear finding also, that students are overly optimistic about their competence and that of their peers. So, an objective assessment of the information capabilities of the constituency is necessary also in the future, and should be 
incorporated to the model, too. 
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